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PATENT 



REMARKS/ARGUMENTS 

The Applicant respectfully requests entry of this amendment, consideration of this 
response and withdrawal of the outstanding rejection. 

By this amendment, claims have been canceled or amended to more clearly 
articulate the invention in response to the final office action. 

The Examiner is to be thanked for the explanation of his interpretation of claim 
language. While the Applicant respectfully disagrees with the logic of the Examiner's position 
and the appropriateness of the cited art, as hereinafter noted, the Applicant also feels compelled 
to explain the theory behind the invention so that the claim language is considered in proper 
context. 

The Examiner has stated that real time processing is a subjective term which in 
this instance the Examiner is interpreting to mean "a process being done before a deadline to 
prevent bottlenecks." The Examiner refers to a statement in Chin reciting that peak performance 
processing of the system therein is 40 billion operations per second, which the Examiner has 
interpreted as real-time processing The Applicant offers a more precise definition and herewith 
respectfully points out that the interpretation of Chin as real time is fundamentally flawed, even 
by the Examiner's definition. A real-time process is one in which computation is during the 
actual time that the related physical process transpires in order that results of the computation 
can be used in guiding the physical process. (IEEE Standard Dictionary of Electrical and 
Electronic Terms) Real-time systems are those that guarantee that the system will respond in a 
predetermined amount of time. (The Electrical Engineering Handbook Sect. 96.6) 

Thus, a processor or processor set may be slow or fast, so long as it guarantees 
that its processing invariably yields a solution to a problem before the processing system enters a 
non-reversible catastrophic state. Chin does not guarantee such a solution, a solution to a 
problem in interaction with the physical world. Thus, the Applicant submits that the term "real 
time" has not been given proper patentable weight in the claims. Moreover, Chin is a non- 
enabling disclosure, and the Applicant can supply reports thereof if required. Such reports would 
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disclose that it is generally known in the art that Chin and systems of its ilk do not work as 
asserted. 

Referring to the invention as generally claimed and now in claims 1 and 10 as 
more specifically amended, namely a system and method that provide image analysis in a real- 
time (i.e., guaranteed-solution) physical environment, the claims recite three distinctly different 
layers of processing that solve at least three distinctly different task sets, now even more 
specifically defined. Layer One is a processing layer that solves the per-pixel processing tasks, 
such as sharpening the image, enhancing contrast and color separation, and enhancing gradients 
and thereby improving the ability of the next layer to distinguish objects. 

Layer Two sits on top of Layer One and works in conjunction with Layer One. 
That means that Layer One and Layer Two must both be present simultaneously for the system 
to provide Layer Two results. Layer Two finds object perimeters and therefore is referred to as 
object-dependent and pixel-independent. It determines object perimeters to find object 
boundaries, and it does not necessarily identify objects. It determines static and dynamic object 
features, such as linear and non-linear motion parameters of all objects in the field of vision. 
Results from Layer Two are used by Layer Three, meaning that Layer One, Layer Two and 
Layer Three must be present to constitute a complete solution to real-time image analysis. Layer 
Three takes the Layer Two results (and of course implicitly Layer One results) and uses them. 
Examples of Layer Three processing is computation of data to present results, tracing back the 
objects' motion to its origin, or predicting the future path or trajectory of the objects in the field 
of vision. Other results can be derived from the extracted data in Layers One and Two, but all 
results can be used in Layer three to present, display or further compute these data sets. Layer 
Three processing can occur in any general-purpose single-core or multi-core RISC or CISC 
processor. As now more specifically recited in claim 1 and as due to the demands of real-time 
processing, Layer One is an MPP SEVID, and Layer Two is an SMP-type matrix multiplication 
processor architecture combined with an SMP-type Fast Fourier Transform processor 
architecture (including for example a DFT architecture). Any layer on top of Layer Three may be 
a single-core or multi-core RISC or CISC processor architecture, with an appropriate reliable 
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operating system. Layers One and Two do not need a general purpose operating system since 
their real-time requirements dominate the needs, and not the ability to manage resources. 

By this response, the Applicant also renews all previous arguments, particularly 
those filed November 1, 2007, and respectfully traverses the prior characterizations of the 
relevance of the prior references. 

The Applicant points out that the present invention must be understood as being 
directed to a new type of architecture. That architecture is not shown or disclosed in any of the 
cited patents in combination. The claimed architecture may be considered to be of the class of 
the so-called Harvard structure. The cited art is all of the so-called von Neumann architecture. 
(Juvinall explicitly states that it is of the von Neumann model.) Chin is a set of von Neumann 
machines that are networked in such a way to maintain inter processor synchronization, or so it is 
asserted. Therefore, Juvinall and Chin do not lend themselves to any obvious combination, and 
certainly not a combination that points to the claimed invention. 

The subject matter of Juvinall, which was specifically cited by the Applicant, is 
the type of prior art with deficiencies that the present invention addresses and overcomes, as 
discussed in the background of the invention. None of the art of record discloses computational 
structures or processing: 1) on a per-frame basis with one processor per pixel , in combination 
with 2) N-way symmetric multi-processing image processing at the object level that is in 
combination real-time capable. 

In the Specification, the Applicant previously addressed the deficiencies of the 
prior art, to which Juvinall and Chin belong and represent. Key features are not disclosed or 
taught: First, processing of data at the pixel level (one processor per pixel) on a per-frame basis 
in a real-time physical environment and, second, processing of data at the object level with an 
SMP in a real-time physical environment, with all the constraints imposed by real-time 
processing. This is a fundamental and economically significant difference between this and the 
teachings of the prior art. 

The deficiencies in the teachings of each of the references are herewith addressed. 
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1 . The Examiner contended that Juvinall discloses that the pixel processor 
array receives image data as a full frame and processes it as such. That is not true. The Juvinall 
system reads the frame data into frame memory, and then the pixel processor array reads them 
block by block (or block-parallel, depending on how many pixel processors are available), and 
then the pixel processor processes the blocks such that pixels are processed sequentially within a 
block. There are a number of fundamental differences between Juvinall and the claimed 
invention. The claimed invention uses one processor per pixel, even if the frame data is 
multiplexed. Juvinall uses blocks. Juvinall describes a 1 -bit processor. The present invention 
has a processor per pixel that uses as many bits as there are in the color representation, typically 
8 bits per color or at least 24 bits per processor. By its nature, the claimed invention can and 
does traverse block boundaries for multiplexed frames. Traversal of block boundaries is not 
mentioned in Juvinall, nor is it required by the process described. Juvinall' s MPP SJJVTD systolic 
array is based on a multitude of traditional CPUs, with I/O and external memory, each requiring 
and relying on common DRAM access for code and image data as well as for swap and 
temporary data. Specifically, Juvinall describes the processing of image data such that data is 
streamed into DRAM of the system, and then this frame data is fetched from DRAM block by 
block into the block RAM within the plurality of pixel processors, and is transferred back from 
there to DRAM (to complete the frame data), and only then from there into the next layer. 

In the present invention, the architecture is such as to allow real-time streaming 
the data from an image sensor or a camera into the first layer processor array, and from there 
directly into the second layer processor within the constraints of real-time processing. A frame 
buffer between the camera and the first layer processor only needs to be deployed in this 
architecture if the entire frame is comprised of multiple frames from multiple cameras, or if 
realignment of frame data is required because the data is streamed in via unsynchronized high 
speed serial links from a high-speed camera (a camera with 10,000 frames per second or more). 

2. Juvinall does not teach that the second layer processing is carried out in an 
SMP. In fact, Juvinall suggests the second layer processor array to be another MPP SMD. 
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3. The Examiner then states that Chin discloses an N-way SMP system to 
perform the second layer processing. Chen does not disclose an N-way symmetric multi 
processor system. The assertion is evidently a hindsight reconstruction based on the Applicant's 
claim - it is certainly not found in Chin's disclosure. Chin very explicitly refers to an MPP 
SIMD and thus cannot be said to read on the claimed second layer. 

It is a major change and not a clarification of the post-processing engine to change 
architecture from an MPP SIMD to a real-time capable SMP. Moreover, the MPP SIMD 
architecture herein claimed is not at all conventional. 

In summary, it is also not obvious to one skilled in the art to use an MPP SIMD in 
the object independent first layer of image processing and an SMP for the object dependent 
second layer of processing and then third layer processing. 



CONCLUSION 

In view of the foregoing, Applicants believe all claims now pending in this 
Application are in condition for allowance and an action to that end is respectfully requested. 

If the Examiner believes a telephone conference would expedite prosecution of 
this application, please telephone the undersigned at (650) 326-2400. 



Respectfully submitted, 
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Kenneth R. Allen 
Reg. No. 27,301 

TOWNSEND and TOWNSEND and CREW LLP 

Two Embarcadero Center, Eighth Floor 

San Francisco, California 941 1 1-3834 

Tel: 650) 326-2400 
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